It has long been known that by suitable photographic processing involving the use of positive and negative forms of an image that certain characteristics of the image can be emphasized or deemphasized by the overlay of various positive and negative images with suitable translational or rotational displacements, density differences and/or spacings. It will be appreciated, however, that processing and display when utilizing strict photographic techniques is time consuming in view of the length of time necessary to develop, prepare and position the positive and negative images. This technique is thus not a real time technique and thus is not applicable to on board target tracking systems in missiles, guided projectiles and various other guided ordinances. Image enhancement has also been accomplished in the past by use of Fourier Transformation of a segmented image with appropriate transform manipulations enhancing or deemphasizing a particular characteristic. This is usually accomplished with a large amount of computer storage and the necessity of storing the value of each point in a given image so that the appropriate Fourier Transform manipulations can be performed. Thus in image enhancement by computer processing, substantially all values in the input image must be utilized in calculating each point in the output image and these values must be stored and appropriately addressed so that the appropriate transform can be applied. With the advent of the Fast Fourier Transform (FFT), computer time has been significantly reduced. However, even with the Fast Fourier Transform algorithm all points in the image must still be sampled and stored, at least once, which still takes considerable time. This technique is likewise not readily adapted to on board target tracking systems. This is because the FFT processing results in a non-real time system for image enhancement in which the final result may take as short a time as 5 minutes for a 500 line TV picture or as long a time as several days depending on the complexity and degree of enhancement required. Note that a 500 .times. 500 line TV picture involves 250,000 elements. A 10,000 .times. 10,000 line picture typical of photographic resolution, requires processing of 10.sup.8 elements.
Holography has also been used with spatial filtering for image enhancement as a hologram involves Fourier Transforms, without need for computation. However, the required photographic development and reprojections prevent these techniques from being applicable to the above mentioned on board apparatus. Also coherent illumination is necessary.
The subject system is a "real time" system which may be utilized on board a guided missile, etc. The system involves two optical channels and a point-by-point subtraction of the two images produced by the two channels. This provides a video difference signal which represents the difference between a positive and negative image of the scene at which the system is pointed. In terms of Fourier Transforms, the transform of the image equals the transform of the object multiplied by the transfer function of the optical system.
In this patent a number of techniques are enumerated that allow synthesis of a wide variety of transfer functions which lead in turn to a variety of spatial filter effects on images. Translational, rotational, density and/or magnification differences between the two images can be optically or electronically introduced to provide, for instance, edge enhancement, size discrimination, enhancement of a particular set of parallel image lines, and peripheral image enhancement or enhancement of the center of the image, sometimes called "boresite" enhancement. It should be noted that while some of the above enhancements may be accomplished by Fourier Transform manipulations, in a computer or holographic manipulations, different effects depending on location of the image plane are not achieveable by such techniques. These enhancements are those which are a function of position in the image plane. Thus, in addition to the real time aspect of this invention, there is the added capability of providing non-Fourier Transform enhancements.
In essence, the subject system provides the equivalent of two optical systems referred to herein as "two barrel optics" in which the receptors of these two optical systems are scanned point-by-point in a twin scan system with the outputs from the scanning apparatus being subtracted on a real time basis, and with the video difference signal then being displayed on a conventional raster scan device. The desired enhancement, or deemphasis, is obtained by controlling the image difference parameter between the images in the optical channels such that the positive and negative images are electronically superimposed in a manner similar to the photographic process.
In one embodiment the subject system is designed for edge enhancement and size discrimination so that an image may be enhanced over background clutter by virtue of its sharp edges as well as its small size as compared to background objects. This may be important in, for instance, picking an aircraft out of clutter involving clouds behind the aircraft. In this case it will be appreciated that the aircraft is much smaller than the clouds. Moreover the aircraft has sharp edges as opposed to the usual cloud configuration in which the edges of the clouds are not as sharply defined. In one embodiment this is accomplished by utilizing a "two barrel" system and by scanning the images produced, with the image produced by one barrel being blurred by receptor offset from the focal plane of this barrel. This is called the "focus-defocus" case. For the present purposes the term "two barrels" refers to two optical systems or channels in which each barrel produces an image. This system involves a "parallel twin scan" in which two parallel moving scanning beams are produced, one scanning one image and the other scanning the other image. The scanning beams simultaneously read out a corresponding element or corresponding location on each of the receptors. As will be discussed later, the same result can be achieved in a one barrel system with appropriately weighted summing or averaging of elements adjacent to the scanned element providing a simulated blurr.
Emphasis of a particular series of parallel lines, with simultaneous deemphasis of orthogonally oriented lines, may be accomplished by a twin scan two barrel system, with one scanning beam being offset with respect to the second scanning beam in a direction orthogonal to the line to be emphasized. This means that while one beam scans a given element in one image the other beam scans an adjacent element in the other image. The deemphasized lines will be the lines in the direction of the scan offset. This same line emphasis/deemphasis can also be accomplished with parallel twin scan and a skewing of the optical axes in the two barrel system with the offset angle in the direction of the deemphasized lines. The skewing displaces the position of one image with respect to the other image to yield the same result as the offsetting of one scanning beam.
The subject system also permits peripheral image enhancement in which circumferential line elements of objects at the periphery of the image plane are enhanced over those at the center. This is accomplished in one embodiment in a two barrel system with a parallel twin scan arrangement, with the two optical systems having slightly different magnifications. In another embodiment radial line peripheral enhancement utilizes a two barrel system with rotationally displaced receptors and a rotationally displaced twin scan system with the rotational offset providing the radial line peripheral enhancement.
Another type of peripheral image enhancement involving line symmetry enhancement at the periphery of the image may be accomplished by a parallel twin scan system with identical optical channels, in which the two barrels have parallel optical axes but the receptors are skewed about the line of intersection of their superimposed receptor planes.
Additionally, orientation independent peripheral image enhancement may be accomplished with the use of a two barrel, parallel twin scan system and field flattening optics at the image plane of one of the barrels, with focal plane coincidence of the two optical systems at the center of the overlapped images.
Central image or boresite enhancement, on the other hand, can be accomplished with parallel twin scan apparatus and a field flattening optics at the image plane of one of the two barrels, with focal plane coincidence of the two optical systems at the outer edge of the overlapped images. The same type of central image or boresite enhancement may also be accomplished by use of a centrally weighted, radially-weakening density filter at the image plane of one of the optical systems.
In general the above systems can be characterized as follows: EQU visual image displayed = .sup.-.sup.1 { f(a.sub.x, y) -f(b.sub.x, y)} ,
where f is a monotonic function, PA1 where a.sub.x, y is the voltage on an image point in the (a) channel at coordinates (x, y), PA1 where b.sub.x, y is the voltage on an image point in the (b) channel at coordinates (x, y), PA1 where G (x-x', y-y') is the point spread function of a point imaged at coordinates (x', y'); as seen at image coordinates (x, y) PA1 where image coordinates (x, y) = M.xi., M.eta.); where M is magnification; and where .xi., .eta. are the orthogonal coordinates of the object in the object plane; and where l(x', y') is the "idealized" image intensity corresponding to the object intensity at (M.xi.', M.eta.'),
In general: EQU a(x, y) = .intg.l(x', y') G.sub.a (x-x', y-y') dx'dy'
The point spread function is the variable in terms of apodization in the lens plane, lens system characteristics or receptor orientation and location in the (a) channel. EQU b.sub.x, y =.intg.l(x', y') G.sub.b (x-x', y-y') dx'dy'
Possible monotonic functions, f, applied as above, may result in images related to the original such as ##EQU1## among others, and since all of these are nonlinear, they may yield types of enhancement that cannot be performed by Fourier Transforms.
It will be appreciated that in all the above enhancement techniques the twin scan outputs are differentially added to give the aforementioned video difference signal which is then presented by a conventional raster scan display. Ratios of twin scan output signals and other functions also provide for a variety of image enhancement possibilities not coverable with Fourier Transform methods and are included as part of the subject invention.
What has heretofore been described involves analog processing by virtue of certain optical arrangements to provide for various types of image enhancement/deemphasis via electronic positive and negative image overlay. However, similar results can be obtained with some time lag by the use of digital processing with a single barrel system through the use of digital processing to simulate translational offset, rotational offset and various adjacent element weighted summing or averaging heretofore mentioned.
The enhanced image, as described above, can always be added to unprocessed positive (or negative) image, with mixtures of the original and enhanced images in any proportion. Aside from what aids such mixtures may provide to an observer, such mixtures can be used for equalization of spatial frequencies analogous to audio equalization in hi fi equipment. For example, if the modulation transfer function of an imaging system trails off with increasing spatial frequency (as it always does), and if signal/noise is sufficiently good, much of the roll - off can be compensated by adding an amplified version of focus-defocus enhancement (essentially with a S.sup.2 low-end roll-off) to the original image, which tends to equalize the optical qualities up to the high end roll-off of the enhanced image.
A further possibility is to combine the original unenhanced image, in black and white, (on a color display) with the enhancement signal, derived as in the above description, converted to a chroma parameter. There are several ways to do this, as the chroma signal provides two degrees of freedom. One way is to: choose the hue, say red, as fixed. Let the overlay signal control saturation (while unenhanced picture controls gray level). Choose the degree of saturation, and let the overlay control hue. Then pick an 1:1 relationship between hue and saturation or between Q&I signals. Then define a path in chroma space and let the overlay control position along this path. The "spare" degree of freedom, at least in principle, permits two independent overlays. For example, using displacement between imagers, and three devices, one can obtain: 1. a straight picture, 2. a left-right displacement, 3. up-down displacement. Now, let the straight picture operate the black and white channel; let the left-right displacement operate the I signal, and let the up-down displacement operate the Q signal. Thus we have two independent overlays presented simultaneously on one screen.
In another embodiment, in a two barrel system, using separate lenses it is possible to have different size and shape aperture stops in each one, e.g. a square aperture in one and a circular aperture in the other. This will tend to emphasize certain shapes of objects, in this example objects with fourfold symmetry and the right orientation. The differences could be even more subtle, e.g. different apodizations in the aperture planes of the two lenses. The apodizations may simply have radial variation about the lens axis; more generally they may have circumferential variation, which would tend to emphasize objects of certain shapes of symmetries as well as sizes.
Aperture stops of apodizing filters can be applied sequentially, in alternation with a single lensing system as well as simultaneously in two lensing systems, if picture motion is not too rapid.
It will be appreciated that a computer or electronically controlled apodizing screen may be utilized in the aperture plane (which can itself be a video image or pattern, say on a liquid crystal display, or on a schlieren medium such as the oil film in a G.E. light valve, where an electron beam "writes" a schlieren pattern on a surface, to be in a lens plane for this application. Rapidly changing and controlled apertures can be formed ahead of one or the other optical channels. In other words, with a 2-D imaging screen capability in the lens plane, different for each lens, one can either "write in" specifications corresponding to the desired emphasis characteristics or, "closing the loop", "lock on" to an acquired image.
It is therefore an object of this invention to provide a real time optical processing system for image enhancement involving the generation of a video difference signal from scanning the images developed by two optical channels, either actual or simulated.
It is another object of this invention to provide improved apparatus for a wide variety of image enhancement/deemphasis results in which edge enhancement, size discrimination, plane emphasis/deemphasis, peripheral image enhancement and/or central image or boresite enhancement is provided.
It is another object of this invention to provide novel image enhancement apparatus and methods involving real time processing and point-by-point treatment of images in which only a pair of image values are necessary at any given period of time for the generation of an enhanced image.
These and other objects of this invention will be better understood in connection with the following description in view of the appended drawings in which: